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ABSTRACT 

This guide for college teachers, written from a 
British perspective, focuses on economic and efficient practices for 
assessing students. The guide groups various practices under 10 major 
strategies: (1) decide whose interests assessment is serving; (2) 
avoid over-sampling the course; (3) avoid over-questioning; (4) avoid 
over-reading student work; (5) avoid over-commenting on student work; 
(6) avoid over-grading work; (7) refine current policies and find 
ways of using present methods better; (8) consider alternative 
approaches to assessment policy and practice; (9) consult original 
sources for how-to-do-ic details. Each strategy is discussed in a 
separate section which begins by presenting some general rules and 
then suggests particular assessment tasks in light of identified key 
factors. Each section also describes and explains various practices 
and practical applications with citations from the literature that 
describe the use or development of that practice. All the specific 
practices were chosen either because the editors were personally 
familiar with them or because the practices have been addressed in 
the educational literature. Practices were chosen because they 
potentially offer greater economy and efficiency in assessment and 
because of their educational value. Contains 113 references. (JB) 
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Teaching with reduced resources 
Preface to the resource guide series 



The late 1980s have seen increasing pressure on institutions of higher education: decreased 
resources for equipment, support staff, teaching space; inflated costs of texts, serials, equipment; 
more students but fewer teaching staff - and so on and on... 

If there was slack in the system, it has been well and truly taken up as the decade draws to an end. 
How will we maintain the high standards of Higher Education into the '90s? 

Staff of the Professional Development Centre of the University of New South Wales (formerly the 
Tertiary Education Research Centre) began seriously and systematically to seek answers to that 
question in 1988. 

a preliminary report was circulated for discussion at a meeting in the Centre. At that meeting 
participants from many Schools of the University were asked to identify particular areas of concern 
which the Centre might address by collecting and disseminating resources. 

It was agreed that the Centre would produce a series of reports addressing the conflict between 
reduced resources and maintaining educational quality. 

The goal of the series is to help academics to manage their new teaching circumstances intelligently 
and creatively. It collects information and advice from a wide range of sources and adopts a 
modestly optimistic view of our current predicaments. 

The series is based on the following premises: 

(1) Some teaching methods could be made more efficient. 

(2) Recent research, leading to new insights into how students learn, may be able to help. 
For instance, if it could be demonstrated that some burdensome tasks, contribute 
little to the quality of learning outcomes, we may be happy to abandon them. 
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(3) Better technologies and improved techniques may help ease some of our heaviest 
teaching burdens, particularly in regard to some of our most regular, tedious and 
mundane tasks. 

(4) Improving efficiency and cost effectiveness need not reduce the quality of student 
learning. 

Obviously, all the ideas summarised in these documents will not apply in all teaching circumstances. 
However, we hope that there will be something for everyone somewhere in the series - a suggestion 
that triggers a line of thought which results in even a small improvement to her or his life as a 
teacher and also to students' lives. 



David Bond 



SCED Paper 78 

ERIC 



4 

G 



Strategies for assessing students 



Resources guide #1 
Strategies for assessing students 



Introduction 



Even under the best of conditions the unending cycle of assignments, tests and examinations 
places very onerous demands on academics. It is easy to feel frustrated, even resentful, at the 
massive amount of time spent on assessing one's students. 

Severely diminishing resources cast an additionally sinister shadow over the assessment picture. 
Growth of class sizes and reductions in ancillary and part-time staff are already placing heavier 
teaching loads on those who remain, with more items of student work having to be handled 
by fewer people. 

Assessment, already a problematic part of the academic's role, is fast becoming the subject of deep 
scrutiny and substantial re-thinking at all levels of university life - from that of the individual 
academic to School and Faculty level. 

The goal of this Guide is to help academics manage the assessment part of their new teaching 
circumstances intelligently and creatively. To achieve this, it offers information and advice 
drawn from the widest range of contemporary and traditional sources available. 

The Guide does not presume to be an exhaustive survey of the current state of assessment theory and 
practice. On the contrary, it is a practical handbook for "dipping into" as the need arises. Its 
references are culled from the files of educational developers who have consulted with academics 
on assessment over many years. 

Our choice of any item for inclusion indicates either that we have encountered it in person 
(at UNSW or elsewhere) or that it has appeared documented in educational literature. 

We choose it because it prima facie appears to have the potential to contribute to greater economy 
and efficiency in assessment and because it sounds like an educationally reasonable thing to do in 
appropriate circumstances. 

We are thus glad to share each item knowing that our readers accept responsibility for 
thoughtfully appraising it before "buying" it from us. And we hope that whenever any new idea is 
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implemented it will be - as all educational recommendations should - closely monitored to see 
whether it produces the desired results. 

Perceptive readers will notice that a few important contemporary texts and studies on assessment 
are not represented in the reference list. That is because the more general surveys and compendiums 
on assessment are less likely to contain the detailed cases and examples typical of what we have 
drawn upon here. 
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How to use the guide 



This Guide may challenge a few of our customary assessment practices, and it will suggest 
ways of modifying others to make them more efficient. 

It will also explain how some new thinking, together with new technologies, may help take the pain 
out of unavoidable assessment tasks that remain. 

The ideas are grouped into ten major strategy sections, within each of which are a wide range of 
suggestions. 

No attempt has been made to rigorously arrange these items within each section. Items are titled 
boldly and informatively, so you should soon be able to find the ones that interest you. Limited 
cross-referencing has been attempted. 

After browsing through the pages and finding ideas that seem to address particular needs or concerns 
of your own, you may then get whatever further information you want by either: 

studying the brief summary - it may be self-explanatory and all you need, 

or obtaining one of the short, readable papers listed at the end of each 
item to explain the idea in greater detail. 
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Strategy 1 

Decide whose interests 
assessment is serving 



First consider whether, as a general rule, more than one party needs, or expects, to get something 
out of assessment processes, 

and, if so, what each party generally needs, or expects, to get. 

Whether, for instance, students expect feedback on their learning success, 

or expect to have their learning improved, 

or expect accreditation via the assessment. 

And consider whether staff expect to get feedback on how effectively they have taught, 

or information about individual students and their learning difficulties, 

or information to help monitor the progress of the course, 

or information to inform future changes to the course or the teaching method. 

And whether the university organisation expects, at the end of the day, to get scores 
and grades for student records. 

Then consider, in the particular assessment task being implemented, which of the needs and 
expectations mentioned above apply to it, 

particularly whether it x: it being conducted for the benefit of one particular party, 

or for one party more than for another, 

and, if more than one party's interests are involved, and resources are few, which 
party's needs should take precedence. 
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Decide whose interests assessment is serving 

After these considerations, the items that follow may provide a source, of ideas for you to use in 
starting to develop a more economical assessment system. 

1.1 Assessment aims 

It is wise to state the aims of assessment in each separate subject as clearly as possible. That 
means asking "Why are we assessing this subject?" and "What do we or our students expect to gain 
from the assessment?" Some assessment in our programs may be found to have no proper rationale 
and therefore be unjustifiable. 

This involves making some important distinctions. The most crucial of these are between 
assessment for final grading and accreditation, assessment for feedback on student progress and 
assessment as a learning tool. 

Very different approaches may need to be taken in each case. Time and effort are soon wasted if we 
start using the wrong approach. 

Having made the necessary distinctions, we are in a better position to judge whether our 
customary assignments and examinations are, in fact, appropriate for the chosen aims or 
purposes of our courses. Some may be inappropriate, involving us and our students in pointless 
labour and waste of resources, hence should be jettisoned and replaced by others. 

Cox (1988) 144-146; Feletti (1988) 147-151; H.E.R.O. (undated) 3-4 

1.2 Student learning goals 

Certain assessment policies may, without our being aware, actually militate against students 
learning in the way we hope or intend. Most academics would, for instance, value their students 
learning the subject through using "deep" rather than "shallow" information processing. 

Research shows that some examining practices may, however, encourage and reward "shallow" 
processing. Boyd and Cowan argue, for instance, that only methods involving self-assessment can 
reliably achieve "deep" processing in university education. (See Strategy 8 below) 

The resource implications are not difficult to grasp. If we were to be using an inappropriate 
assessment policy (vis a vis the type of student learning we value) then all of the time and other 
resources we spend on Cat assessment may be contributing to a learning outcome that is quite 
contrary to what we intend. Our assessment practice would thus be negating the value of many of 
the resources previously invested in designing and teaching the course itself. 

Boyd and Cowan (1985); Ramsden (1987) 
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1.3 Communicating essay assessment criteria 

Whatever the purpose of a particular piece of assessment, students need to know the criteria by 
which their effort will be judged. Communicating these criteria is an essential prior stage to the 
assessment act itself. 

There are, of course, many ways of doing this communicating, most quite simple and obvious. 
Criteria forjudging essay work, however, are notoriously more difficult to communicate than . or 
other types of task. 

One effective method of communicating standards and criteria for essay assessment is the "Multiple 
Choice Essay**. Students* essay scripts from previous examinations are kept, with permission, for 
use in this technique. Alternatively the teacher may write the scripts. 

Five essays are chosen, to represent each of five grades or each of five levels in order of merit. 

Students are given these to read and to rank in merit order. They submit merely their rank listing, 
from which a rank correlation coefficient is calculated, indicating the degree to which their ranking 
matches that of the teacher (or an experienced examiner). 

Marking this test requires no reading whatever by the examiner. The content of the test can be 
varied at will by using different sets of answers. It is claimed to produce remarkable improvements 
in subsequent examination performance and is of particular value as a formative assessment 
technique. 

Fox et al (1985) 13-15; Gibbs et al (1986) 131-142 

1.4 Communicating MCQ assessment criteria 

Criteria for assessment of subject matter by Multiple Choice (and other objective-type) Questions 
are conventionally communicated through statements of Learning Objectives. 

In some subjects it is appropriate to use statements of desired student behaviour as these objectives. 

Davits (1976) 91-95 
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Strategy 2 

Avoid over-sampling the 
course 



First consider whether, as a general rule, every assessment task you set needs to be treated in 
the same manner - having the same time invested in it and the same amount of attention and 
involvement on the part of the teacher, 

and whether all need to be read equally fully, 

or commented upon to the same degree, 

and whether every student's work needs to be appraised in each case, 

and whether grading or ranking has to be an outcome in each case, 

and whether every piece of work always has to contribute towards continuous assessment. 

Then consider, in the particular assessment task being implemented, 

whether the nature and scope of the task has been pitched at a level that minimises resource 
demands, consistent with achieving whatever purposes you have in mind in setting it. 

For instance, whether this task could contribute to student learning if it were not graded, 

or not commented upon, 

or not even read by you, 

and whether you could give some useful feedback on this task without having to grade, 
or without studying it completely in detail, 

and whether you could get enough feedback from this piece of student work without studying 
in detail every student's work in the class. 
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After these considerations, the items that follow may provide a source of ideas for you to use in 
starting to develop a more economical assessment system. 

2.1 Assessment load on students and staff 

Assessment demands have a major impact on both student work-load and on staff resources. It is 
often possible to reduce the number of tasks for assessment without detracting from the rigour of a 
course or diminishing the reliability of final assessment grades. 

In a study of assessment loads at Macquarie University large variations were found from course to 
course, and some subjects were able to be pin-pointed as clearl) being over-assessed by comparison 
with others. 

We should try to estimate the number of hours spent by staff in setting, explaining, supervising, / 
collecting, marking, grading, processing scores and giving feedback, all relative to the number of 
contact hours of the course. 

A similar exercise can be carried out in regard to student loads. There we can compare the number 
of hours typically spent being briefed upon, carrying out, and receiving feedback on assessment 
tasks, relative to the total contact hours of the course. 

Subjects with anomalous assessment loads on either students or staff become, by definition, cases 
for suitable attention. 

Macquarie University Education Committee (1984) 4-5, 19-21 

2.2 Predicting possible student responses 

It is wise to try to anticipate where students will take the questions we set - essay questions in 
particular. Some instructions, some questions, may subsequently make our task as markers much 
more difficult than we ever intended. It is obviously best to discover this fact before students go to 
work on it. 

This involves taking time to put ourselves in a student's position and imagine some of the possible 
misinterpretations of our question and what those might lead to when we finally get the piece of 
work back for marking. This can be difficult when it concerns questions we have ourselves set and 
other staff can often give the best insight into ambiguities and lack of precision in our instructions 
and question-wording. 

Nightingale (1986) 18-27 

2.3 Setting and marking group tasks in class 

Provided group work is compatible with the aims of assessment in a particular case, it can often be 
most efficient and economic if group assessment tasks are both done and marked in class time. 
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A seminar group, for instance, can handle a question in class by collaboratively drafting note-form 
or outline answers. The result can then be appraised in class. 

Either approach can lead to better discussion and clarification of writing or problem-solving 
strategies and marking standards. 

Gibbs etal (1986) 103-107; Nimmo (1977) 186-187 

2.4 Non-assessed "prior stage" tasks 

These refer to any kinds of tasks that may lead up to the major one. Where prior tasks of this kind 
are set, it is often reasonable to grade only the last, and the earlier ones may not need to be assessed 
at all in the conventional way. 

For examples of these prior-stage tasks, we siiould consider - among other more obvious types - 
the use of course journals, reading logs, and reflective pieces of many different kinds. 

Nightingale (1986) 10-13; Zubrick (1985); Gibbs etal (1986) 115-117 

2.5 Lead-up tasks that can be self-marked 

Assessment that is "formative" in function - ie not required in order to give a final grade - can 
often be carried out on lead-up tasks, such as synopses of proposed essays. 

In such instances it should be remembered that the appraisal of these does not necessarily have to 
be done by academic staff. Students can give each other constructive feedback and learn a lot as they 
do so, provided the task is designed and set with this option in mind. (See also the sections on Self 
and Peer Assessment in Strategy 8 below) 

Paxton (1976) 

2.6 Watching out for essay "overkill" 

Short answers, lists, synopses and outlines can be used to replace extended essays in many cases. 
If we critically inspect what really needs to be assessed in our subject, this might be only a 
portion of what would be demonstrated in a full, completed essay. 

We need to ask why, then, we should have to mark a full essay if much of what it reveals is 
superfluous to our present assessment needs? 

2.7 Giving a choice among essay-type exam questions 

The main arguments against giving a choice among essay-type question options in tests and exams 
have nothing to do with economics or efficiency in the first instance. They are that, according to 
research, if a paper has say 12-15 questions of which only 5 are to be chosen, the chances are that: 
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(i) the different options will all vary in difficulty, 

(ii) marker subjectivity and unreliability will be greatly amplified, 

(iii) many students will choose unwisely among options, not doing themselves justice. 

Hence on all empirical counts the practice of giving a choice tends to make exams unreliable and 
unfair. Each student in fact sits for a different paper. 

On the issue of resource efficiency, commonsense indicates that we have less work to do if: 

(i) fewer questions need writing when the paper is being set 

fewer marking schemes need designing during the marking stage. 

In addition, markers generally find it more efficient to mark more answer*, of the same question 
than a few answers of many different questions. 

So, if you think you would find it easier to set papers comprising a set of common questions with 
no options, you have both research and commonsense on your side. 

Students, accustomed to being given options, may resent the change as unfair, so you may 
need to marshal your facts and explain clearly your reasons for the new policy. It will be a fairer 
exam for them and the marks you award will be more reliable. 

Stanton (1981) 

2.8 Diversity in assessment strategies 

Student surveys often come out strongly in favour of a diversity of examining methods because 
of the very reasonable belief that only in this way can a "fair deal" be assured. 

Apart from any intrinsic value in using a diversity of assessment methods, a policy of 
encouraging diversity may also be consistent with gains in efficiency. 

Whether this is so will depend, however, upon which particular formats are included in the package, 
since not all methods need be as time-consuming as others in setting and marking. For example, 
an economically diverse package may need to comprise only one long written task, supplemented 
by a variety of shorter, more easily marked tasks. 

As an instance, the examination structure in one professional medical course involves: 

(i) a written paper using the modified essay question style, 

(ii) a multiple-choice test, 

(iii) a simulated diagnostic interview and 

(iv) a practical test. 



, Beard (1970) 193-196 
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Stategy 3 

Avoid over-questioning 



First consider, as a general rule, 

whether the full range of possible assessment setting economies have been considered within 
your School, Department or subject team, 

and whether the fullest and clearest communication has been achieved among staff, and 
between staff and students about those policies. 

Then consider, in the particular assessment task being implemented, 

whether the scope of the course material being assessed could be limited to, 

a smaller section, 

or fewer topics, 

or restricted to a narrower range of skills or competencies that would be easier for you to 
handle when marking, 

and whether the amount of extended writing involved in the task could be limited, 

either by combining, or replacing, written work with short answers, 

or else by using note-form, frameworks or skeleton-answers, 

and whether this material really needs to be assessed at all. 

After these considerations, the items that follow may provide a source of ideas for you to use in 
starting to develop a more economical assessment system. 
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3.1 Reasons for setting particular tasks 

Instead of setting assignments merely because it is customary or expected, start to examine 
critically why the task is being set, what its purposes are and what both we and our students are 
expected to get out of all the effort of setting, writing and marking it. 

While perhaps agreeing with this recommendation in principle, staff often find difficulty in 
finding simple, straightforward ways of carrying it out. 

A simple strategy is to use an assignment scrutiny check-list. In Roe's sample check-list eighteen 
questions are suggested as worth asking ourselves before we go ahead with setting an assignment. 

Once devised, such a check-list can be used, with perhaps small modifications, for a number of 
years - even for the life of a course. 

Roe (1975) 222-223 

3.2 Matching question styles to learning goals 

The styles and formats of questions we set should be matched as closely as possible to the 
learning goals we expect students to be working towards. Teaching for factual recall will require 
very different assignments and assessment tasks than if the learning goal were, for instance, 
problem-solving skills. 

In testing factual knowledge, short-answer, multiple-choice, or some other "objective" -type 
question format may be economical and appropriate. They can be marked and graded with 
minimal effort. 

Indeed it would be wasteful to try to test factual or informational content by using longer question 
styles that were not only inappropriate but more time-consuming to read and mark. 

It is no secret, however, that good "objective" questions do require skill and effort to set in the 
first instance. On the credit side they can be stored in question-banks and re-used, and this is an 
economy that in the long run balances the initial outlay. 

On the other hand, problem-solving or other higher intellectual skills almost always demand 
question formats that take longer for markers to read and appraise. It is possible that in trying 
to economise through converting to objective question formats we may seriously risk abandoning 
the learning goals we claim to have set. With such learning goals as these, if economies are to be 
found, they will generally lie in other directions, and some of these are discussed in items that 
follow (eg 3.9, 3.10). 

Heywood (1977) Chap. 10; Thomas & Bain (1984); Ebel (1972) 144; Hall & Cannon (1975); 
Clift & Imrie (1981) 69-88 
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3.3 Marking advantages of objective tests 

When considering whether or not to economise by changing all or part of a subject's assessment 
to an objective question format, it is important to make this decision in terms of the 
appropriateness of that format to the particular subject matter and the student learning goals of the 
course. Well-designed objective tests have been shown in many studies to be at least as valid as other 
types of question formats provided what we are measuring is: 

(i) factual recall, 

(ii) the application of principles (rules, procedures or formulae), 

(iii) the interpretation of data (exercising judgement). 

If we are confident that they are appropriate for use, their main economic advantage is the saving of 
time at the marking stage. Several fast marking alternatives are available. With computer facilities 
100% accuracy and high speed are available. On Lie other hand, carefully drawn manual sconng 
sheets can be used at a speed determined solely by the speed a marker can add a column of ticks. 
Finally, students can - if it suits your course aims - easily and reliably mark their own tests. 

Whether or not this marking economy warrants the considerable investment in test development 
depends upon their scale of use. 

One user has calculated that, taking into account normal design and validation time, over 50 
students per year over at least 3 years are needed to recoup the initial investment on a test. 

Bell (1985) 

3.4 Communicating assessment requirements 

When students seriously misinterpret what is being expected or intended in an essay, extended-type 
test or assignment, their own time is being wasted in doing the task, and staff time is wasted reading 
and attempting to mark the material. 

Studies show, however, that the essay - and assignment-writing strategies which students devise in 
isolation may not at all necessarily correspond to what is intended or desired by staff. 

Staff should, therefore, adopt - from Day 1 of a course - a general policy of more open and explicit 
communication with students about all aspects of assessment, and not only about the criteria for 
judgement as recommended earlier. 

We need to communicate especially clearly the precise requirements of the Department and the 
range of different approaches that can be taken in order to satisfy those requirements, while at 
the same time accommodating particular styles preferred by individual students. 

Hounsell(1982); 

Branthwaite et al (1980) 105-107 
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3.5 Achieving a coherent assessment policy 

Developing a fully coherent and comprehensive statement of faculty, school or departmental policy 
on assessment is a substantial undertaking, and may require considerable initial investment of 
resources. 

Once such a policy statement is achieved, however, a great amount of the wrangling, uncertainty 
and going over old ground that takes place at examination meetings can be eliminated completely. 

What such a policy might look like is best indicated by examples, and the University of 
Newcastle's Medical Faculty policy is worthy of close inspection as a model. 

It is noteworthy that special care has beprftaken in that policy to "promote the educational merit of 
assessment". 

EngeU Feletti & Leeder (1980); Feletti & Colditz (1979) 

3.6 Reducing the number of major assessment tasks 

Instead of a number of substantial pieces of writing in a subject one may decide, for economy, 
to set only one major work. But in this case we should design a sequence of prior tasks which will 
help students to produce that one major work, eg annotated bibliography, preview. 

These prior tasks may be assessed if it is wished, but in any event won't take as long to deal with 
as extended pieces. It often makes sense for the first piece submitted to be purely a learning 
exercise and therefore not assessed at all. 

Nightingale (1986) 7-9; Bamford (1978) 95; Ebel (1972) 186 

3.7 Note-form essays 

The economy gained in reading-time if students write note answers is obvious enough. The 
less obvious matter is choosing under which conditions a note-form essay or answer may be 
educationally valid. 

Note-form essays can comprise legitimate and defensible assignment or assessment tasks, provided 
the main learning goal is recall of information, or simple understanding of concepts, or recognition 
of what is significant about a topic. 

They are not necessarily inappropriate when other learning goals are involved, but in those cases 
the implications of their use should be studied carefully before implementation. 

Gibbs et al (1986) 23-24 
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3.8 Outline answers 

A variation on the note-form essay is the outline answer. Outline answers can also be quite sufficient 
to show how a problem has been solved, particularly if part of a larger paper containing a 
comprehensive range of questions has been set covering the field of the course. 

Outline answers are much more easily marked than extended or full answers, and provided broad 
coverage has been ensured, they can be a better indicator of overall subject proficiency than a 
single extended task on a narrower topic. 

3.9 Shorter-answer formats in problem-based learning 

Performance in problem-based courses can also be assessed using short-answer or mini-essay 
formats, instead of large pieces of extended writing. 

The shorter written questions most appropriate in such courses often have as their goal: 

interpretation of evidence, 
constructing a design, or 
generating a hypothesis. 

Gibbset al (1986)19-22,25 

3.10 Modified essay questions (MEQs) 

A specially developed shorter written form is the Modified Essay Question (MEQ). This style 
provides a means for specifically appraising problem-solving skills without having to read and 
mark extended written work. The Faculty of Medicine at the University of Newcastle has 
investigated how MEQs can best be developed and used. 

The MEQ format first involves the presentation of a body of information which defines a 
particular "problem" to be solved or decision to be reached by the student. This is followed by a range 
of possible questions to be answered before the problem can be solved or the decision made. These 
may require the student, for example to: 

generate a hypothesis, 

list further information needed before making a decision, or 
outline practical steps or further investigations to be taken. 

Numerical data, pictures, graphs, etc or else "realia" (pictures, charts, x-rays, models, maps, 
printouts etc) are provided in the examination room, enabling the MEQ format to be used to test 
students* discrimination skills and analytic abilities. Knowledge and understanding are then 
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drawn upon to provide an answer that is concise and focused, hence quicker to read and more 
reliable to mark than the open-form essay answer. 

Knox (1975); Hodgkin &Knox (1975); Feletti et al (1980); Feletti & Smith (1986) 
Feletti & Engel (1982) 

3.11 Variations on the standard essay 

Role-play essays and structured essay questions are variations on the standard essay which 
may more efficiently address the particular learning expectations of some courses. They generally 
allow sharper focusing on the specific intellectual and communicative skills students are expected 
to have developed. 

A question of this sort may, for instance, place students in an imaginary situation and ask for 
discussion of how some decision would be made, or invite them to write a particular letter, draft a 
journal article, prepare a ministerial reply for question-time, interpret a body of evidence, or 
design some artefact. 

Experience shows that ii is often possible to design a question along these lines that substantially 
reduces the marking time compared with a more diffuse open-ended essay question. 

Gibbsetal (1986)11-18 

3.12 Scaied-down laboratory reports 

"Instant" reports can be written during a Science or Engineering laboratory session and handed 
in immediately for checking. This tends to keep them brief, concise and concentrated on the 
essentials. 

If it is felt that quality of presentation suffers because of lack of time, uncompleted handouts can be 
used for the reports, in which case the final report simply consists of the handout fully completed. 

Gibbsetal (1986)123-124 

3.13 Pencil-and-paper tests in laboratories 

These include virtually any of the long- or short- answer types of questions discussed earlier, . 
applied to laboratory work. Such tests, in addition to having the highest reliability, are reported to 
be more economical than their competitors such as extended written reports, practical tests, 
observations or interviews. 

Although the construction of a good test is initially time-consuming, its ease of administration 
and marking plus its re-usability make it a cheap form of laboratory assessment. 

Boud, Dunn & Hegarty-Hazel (1986) Ch 4 
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3.14 Shortened examination papers 

Shorter exams, containing fewer questions, are a plausible and attractive solution to the reduced 
resources problem. Teachers should, however, be aware that simply shortening an examination may 
by-pass the important question of what is the intention of the examination. 

If that intention is to adequately sample the course content, there may be lower validity 
associated with lesser sampling, as the number of questions is induced. Whether the shorter exam 
is then justified is a question that can only be answered by referring to the overall aims of the 
assessment program (see Strategy 1 above for discussion of this issue). 

Hill (1978) 

3.15 Multiple-choice plus machine-marking 

We may always set Multiple Choice Questions (MCQ) where "objective" testing is appropriate 
(eg factual recall and conceptual understanding). AnMCQ comprises a stem and four or five options, 
all but one being inconvct answers (distracters). 

MCQs are always best designed by panels and should accurately reflect the specific objectives of a 
course. Guidelines for construction must be adhered to and these are readily available. 

Whenever used, MCQs should be monitored for discriminating power and difficulty, as described 
in Dunstan*s booklet. 

Bandaranayake & Cox (1988) 152-156; Cox (1988°) 157-160; Cox (1988 b ) 101-165; 
Dunstan (1980 b ) 4-6. 

3.16 Saving tutorial time with MCQs 

A useful variant on the regular use of MCQs can lead to saving tutorial time at the stage of giving 
students feedback on their work. 

This is done by linking "wrong" choices (distracters) on such tests to particular page references in 
a set text. Students can then consult the text and find out for themselves why it was wrong to 
choose that option. 

Alternatively, written solutions to test problems can be published in a handout and given to 
students when they submit their test to be studied in their own time. 

Hudson & Hudson (1981); Rees (1986) 
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3.17 Recycling MCOs through "banks" 

It is well known that substantial time and effort are required to develop a good collection or 
"bank" of reasonably valid and reliable objective tests. 

The purpose of such a bank is to provide different sets of questions around particular syllabus 
topics for use in successive administrations of a particular examination. 

The resource investment in an item bank is, however, likely to be worth all the trouble, mainly 
because of the efficiency with which questions can then selected and used on demand. 

Bandaranayake (1988) 166-169 

3.18 MCQ marking methods 

One great economic virtue of using MCQs is that they may be quickly and accurately marked by 
non-subject-experts or by machines. 

For a short, clear explanation of the technology of computer analysis of objective tests, see Patrick 
Squires' paper 

Dunstan (1980 a ' b ); Squires (1975) 

3.19 Variants on standard MCQ formats 

There are numerous variants on the conventional MCQ not always mentioned in the standard 
texts on this method. These include multiple-completion, assertion/reason and best-answer types. 

By using a composite question mode we may be able to combine some of the benefits of easily- 
marked "objective" testing while avoiding some of the limitations of the conventional MCQ 
format - yet still saving reading time compared with an essay. 

Gibbs et al (1986) 41-47; Bandaranayake & Cox (1982); Cox (1982 a ); Cox (1982 b ); 
Litster (1987) 

3.20 Saving test development time with short-answer 
items 

It may be useful to try some of the wide range of easily marked objective test formats other than 
MCQ and its variants. These are readily available and include right/wrong, short answer, 
completion, true/false, matching, and best-answer types. These question types are often 
appropriate for most of the uses to which Multiple Choice Questions are put, but have the advantage 
of requiring generally much less time and effort in development 

Gibbs et al (1986) 27-40 
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3.21 The possibility of re-using tests 

The re-use of tests is particularly appropriate in "mastery learning" courses where fairly 
specific learning objectives are set and students are expected to work towards a particular degree 
of mastery of each part of the course before receiving credit and perhaps before moving to the 
next part. 

Some teachers stop using mid-term and final examinations in these courses in favour of shorter, 
more frequently administered tests. Evidence suggests that such testing strategies lead to greater 
retention of the material studied. 

While there is not necessarily any overall economic or efficiency gain in such policies, a possible 
advantage can enter if the same tests can be re- used by the same group of students. 

Though evidence is somewhat unclear, reports suggest that test re-use can have both cognitive 
and attitutiinal benefits for students. Its implementation, assumed to be economically 
advantageous, may therefore also be educationally justifiable. 

Cotes (1982) 

3.22 Question-generating computer software 

Some subjects lend themselves to the computer generation of test papers - both "objective" 
and problem types, whose main economic advantage is that they can be entirely self-administered 
and self-marked. 

A common type of program takes a single question and substitutes an endless range of different 
values for variables, permitting each student to receive a unique version of the question, thus also 
helping ensure each student's work is authentic. 

Software of this kind is already in circulation, or available commercially, for some subjects. Staff 
with programming expertise or access to authoring programs often write their own. Once installed, 
this software can supply endlessly varied questions for periodic class tests, or for individualised 
self-testing by students. 

The best software of this sort also keeps records of each student's progress through different 
levels of difficulty. This approach is popular for providing self-evaluation revision tests. Students 
find such tests particularly "non-threatening" provided the results are not used to contribute to final 
grades. 

Osborne & Potter (1979); Gibbs et al (1986) 71-75; Andresen (1979) Item 2.1/3; 
Machin & Murray (1985); Turner (1985) 
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3.23 Machine-marking MCQs using microcomputers 

Although on some campuses (such as UNSW) a service exists for computer-marking of MCQs 
provided machine-readable answer sheets are used, it may sometimes be advantageous for tests to 
be marked using microcomputers. 

An optical card reader was originally developed for use with the Apple-II system, and cost 
less than $1500. 

Advantages of such systems are that: 

(i) the user can program unlimited variations on the marking and feedback systems, 
such as awarding bonus marks and taking guess factors into account, 

(ii) item difficulty and discrimination measures, most important for test development, 
can be built in at will, and 

(iii) full personalising and tailoring of printed feedback to students is a simple matter. 
Priestland (1985) 
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Strategy 4 

Avoid over-reading student 
work 



First consider, as a general rule, 

whether staff who teach a course must always be the ones who read students' tests and 
assignments, 

and whether, when student work is read, every part of each assignment needs reading at the 
same detailed level, 

and whether all tasks in a course need to be read as analytically as one another, 

and whether each student's work needs to be read each time a task is set. 

Then consider, in the particular assessment task being implemented, 

whether reading selected parts of this piece of work might be enough, 

or whether reading work from part of the class, instead of every student, might suffice, 

or whether you might read this piece for impression, rather than analysis, 

or whether on this occasion students could read each other's work 
(if you arranged things appropriately), 

or whether they could read their own work 
(if you arranged things appropriately). 

After these considerations, the items that follow may provide a source of ideas for you to use in 
starting to develop a more economical assessment system. 
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4.1 Reducing double-marking 

An instance of inadvertent waste of marking time and effort occurs when normal policy 
dictates "double marking" of essays or assignments. This is often done when work counts 
towards important final grades or prizes. 

Unless the two markings are entirely independent, the second marker having no knowledge 
whatever of the first marker's responses, double marking contributes little at all towards 
reliability and discrimination. In such cases the double -marking policy can be abandoned without 
loss. 

If mandatory double-marking must be retained, it can be made more efficient while economising 
on resources if it is restricted only to pass-fail cases or those that are "borderline" in some respect. 
These are those where high reliability has its greatest practical importance. 

Whatever the case, the use of a "second marker's sheet" is recommended. This serves the purpose 
of easily and quickly transmitting necessary contextual information about the student from 
the first marker (normally the student's teacher) to the second (normally an outsider). 

The effect of this is to ease the second marker's burden and help avoid unnecessary 
disagreements between markers which otherwise waste time in resolving. 

Cockburn & Ross (1978) 39; Hill (1977) 101-102; Murphy (1979) 77; Gibbs et al (1986) 109-111 

4.2 Formative assessment via computer 

One of the most significant developments involving the use of computers in assessment has 
been through the design of software that enables students to more successfully and satisfyingly 
complete coursework assignments in technical subjects such as engineering. 

In those subjects the very complex problems needing to be solved are typically accompanied by a 
severe computational burden. The computer programs in use are those that apply numerical 
modelling techniques to complex structural problems. 

When computer software is adapted to such coursework assignments, experience shows that 
students can become better motivated to tackle tasks individually rather than copying from better 
students' work. 

Moreover, staff are spared the burden of laboriously checking computations in order to locate 
numerical and logical errors and appraise their significance in terms of student learning. Provided 
a full range of software is acquired initially, the supervision required from staff can subsequently 
be minimal. 

Brohn (1986) 
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4.3 Self-assessment questions (SAQs) 

Teachers do not need to be involved at all in some assessment tasks. Self-assessment questions 
(S AQ) of the sort used by the Open University in their printed materials are compelling evidence of 
this. Such questions are not limited to teaching at a distance. On the contrary, SAQs can be valuable 
economic adjuncts to on-campus assessment. 

In a face-to-face teaching environment, the SAQs would normally be supplied via handouts or in 
course notes which students would take home to work on independently, marking themselves 
after completing the task. 

Evidence on the use of SAQs indicates that they encourage students to actively ani thoughtfully 
reflect on what has been studied. Answers - or instructions for how to easily find them - must of 
course be provided for all questions, together with remedial advice for those students who 
may need it. 

This is best done through supplying check-lists, model answers and explanatory discussions. 
However, these all require time and resources for initial development. 

Nevertheless, as in the case of MCQs, a "bank" of SAQs can be gradually developed over time and 
re-used whenever the course is run. The more often such a bank is then used, the greater the return 
on the original investment. 

There is no need to imagine using SAQs for all courses at the start. Try developing them just for one 
course first, or maybe just one section of a course, whatever present resources will permit. 

Gibbsetal (1986) 167-168 

4.4 Guidelines and check-lists 

Whatever the subject, it is important to supply clear written guidelines on how to write essays or 
perform assignment tasks in that particular subject. As well as their direct educational 
benefits, such guidelines help reduce the time-wasting incidence of students unwittingly doing 
a task in a completely inappropriate way. 

Each piece of student work that misses the point nevertheless requires reading. Hence if we can 
reduce the incidence of gross question-misinterpretation this will eliminate some of the time 
otherwise wasted during marking. 

Guidelines also help reduce the incidence of major style or format errors, enabling marker 
time to be more profitably concentrated on content appraisal. 
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Guidelines ca^ cover: 

what is expected, 

style of answer, 

errors and pitfalls to avoid, 

how to approach the task, 

levels of restriction, and 

degrees of freedom available. 

Examples showing how guidelines may be constructed are readily available in the literature. 

Clanchy (1985); Clanchy and Ballardf 1981); Nightingale (1986) 23-7; UTRC (undated) 3.3-3.4; 
Martin (1979) 54-55; Bamford (1978) 

4.5 Marking in laboratories 

In the appraisal of laboratory records two conditions can ease the marking burden. These are: 

(i) stating clear criteria concerning what is required and how it is to be recorded 

(ii) delivering explicit training in how scientists keep laboratory notes. 

If these conditions ire met, the first economy arises because the resultant better quality of notes 
makes for easier teacher-marking. 

A second economy is available if notes are then shared and discussed in small student groups during 
the last ten minutes of a laboratory class. 

Gibbs etal (1986)125-127 

4.6 Computer analysis of laboratory data 

In technical courses where items of assessed coursework require the analysis of laboratory data, 
the limited time available in the laboratory often means the data are incomplete or cannot be 
completely analysed before the student has to leave. This situation is uneconomic and 
unsatisfactory. 

Things can be improved substantially by the use of computer programs that require each student 
to enter and check experimental data themselves before leaving the laboratory. 

The same software can also contain a range of test items on the topic being studied, to which 
completed solutions are supplied, with brief explanations, and an individual score printed out 
for the test. 

Brohn (1986) 
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Strategy 5 

Avoid over-commenting on 
student work 



First consider, as a general rule, 

whether particular kinds of comment might be called for by different kinds of assignment 
and examination tasks, 

and whether there may be an optimal amount of comment that will serve each different 
purpose of assessment best, 

and whether it is possible to clarify the purpose of comments in each different case, 

and whether everything that could be commented upon in a piece of work necessarily 
deserves comment on each occasion, 

and whether the achievement of economies in commenting is a goal worth aiming at in 
your subject. 

Then consider, in the particular assessment task being implemented, 

whether summary comments only, delivered orally to the class, might suffice, 

or whether taped comments, rather than written, might do the job as well, or even better, 

or whether structured feedback sheets might do the job, needing only to be ticked by you, 

and whether there is any role students might play that will 1 educe the amount of commenting 
you will need to make. 

After these considerations, the items that follow may provide a source of ideas for you to use in 
starting to develop a more economical assessment system. 
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5.1 Using students as proof-readers 

In a growing number of subjects, student writing is being taken seriously and incorporated into 
the total teaching program of the discipline. 

Experience shows that this process of integration is enhanced if students are taught to check and 
proof-read their own work before they ever submit it. In this way students come to accept 
responsibility for their own grammatical problems. 

Proof-reading check-lists are an appropriate technique for doing this and examples of check-lists 
are readily available. After an exam has been completed some teachers give extra time for 
students to apply the check-list before submitting their work. 

Hunt (1981) 

5.2 Assignment attachments 

Structured assignment attachments can speed up the marking process considerably. All these 
require is for a box to he ticked, though additional comments can be written if desired. The time- 
wasting repetition of standard common comments can thus be completely avoided. 

Attachments can also achieve other useful goals. They help achieve consistency among multiple 
markers and, if distributed when the assignment is set, they help students understand the criteria 
for assessment. If accumulated over time they can then enable students to better monitor 
their own progress. 

Examples of attachments are available and should be carefully inspected. They demonstrate the 
variety of attachment styles available and the many useful purposes they can serve. 

Examples include: 

science excursion reports, 
sociology book reviews, 
politics and history essays, 
biology and geography assignments. 



McDonald & Sansom (1979) 45-55; Nightingale (1986) 33-37; Gardner & Abraham (1978); 
Murdoch University! ESTR (undated) Assignment Attachments; 
YEAST (Macquarie University) #3fi6 
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5.3 Feedback guidelines 

Explicit guidelines for giving feedback comment can cover whatever strategies a School or 
Department believes will contribute to efficient and fair marking in that subject. 

Feedback guidelines may include, for example: 

recommending that we restrict comments to two or three major errors at most, rather than 
overwhelming students with complete catalogues of shortcomings; 

recommending that we avoid "catch-all" comments that waste time and communicate 
nothing; 

a policy for how errors of form (spelling, grammar, bibliographic referencing, etc) will be 
dealt with; 

a statement about the relative importance of errors of form compared with enrors or 
deficiencies of content. 

If used, however, it is essential that the guidelines be followed by all markers, without exception. 
Bartley I Stanton (1982) 

5.5 Focused feedback 

At the point when v/ork is being submitted for assessment, students can be asked to write a 
note at the end of their work indicating which particular aspects of the work they would like 
feedback on. This practice helps focus the marker's written comments and makes it more likely 
that the comments will be addressed to real concerns of the student. 

Stanton (1988); Oxford Polytechnic (1987) 11-13 

5.6 Student hand-back 

Staff can often feel they are wasting time when, after writing constructive feedback comments 
on a test or exercise, they discover that these are neither read nor attended to by students. 

Therefore, if we regard feedback as important, any strategy that gets students to attend to it will 
contribute, not only to their learning, but to the resource economics of our teaching task. 

One solution to the problem is for students to work in pairs in a tutorial class. Marked work 
being returned with comments on it should then be given, not to the student who wrote the work 
but to their partner in the pair. 
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Then each student, in turn, reads aloud the comments on his/her partner and gives the other a 
peer-tutorial based on the lecturer's feedback comments. 

Stanton (1988); Oxford Polytechnic (1987) 11-13 

5.7 Audio-taped feedback 

Tape-recorded assignment comments are a proven and efficient means of giving feedback and 
their use enhances the likelihood of students attending to and responding to the feedback. 

Students hand in a blank (re-usable) audio-cassette with their assignment and this cassette is 
returned with the graded work. 

Whilst marking, most academics find it quicker, as well as far more personal, to speak their 
comments on students' work rather than to write them on paper. Spoken comments are also likely 
to be more spontaneous in tone and focused more directly on the student as an individual learner. 

Siegel (1985); Cryer & Kaikumba (1987); Nightingale (1986) 38-39 

5.8 MCQ computer-generated feedback 

Program-generated computer feedback can be supplied to students, but generally from MCQs 
only. The TIPS (Teaching Information Processing System) is one such system developed at. 
Duke University and adapted for use in several UK universities. 

In TIPS the test item writers also design tutorial comments which provide the basis for answering 
the item correctly. The computer program then selects and issues to students appropriate feedback 
statements from this "comments item bank". 

The Educational Testing Centre on the UNSW campus operates another system by which 
feedback letters can be sent to students whose tests have been computer-marked. Such a letter 
would typically contain the mark gained in relation to other students, plus 
performance on the test in terms of topics, with appropriate references for remedial work. 

Gibbs et al (1986) 77-78; Ruddick (1985); Dunstan (1980°) 

5.9 Assignment feedback via computer 

Report-generating computer programs are available which give computer assisted fr edback on 
suitably structured assignments. 

These comprise a simple "expert" system enabling the teacher to provide two typed pages of 
individualised feedback on each student assignment or report. 
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Such programs are objective in the methodical assessment of each report according to a 
consistent set of programmed criteria. Nevertheless they allow individual markers to supply 
their own comments at will. This results in more consistent maridng and delivers extensive 
feedback to students despite minimal time being spent by teachers in writing and composing. An 
example of such software is a package developed by Marshall. 

In Marshall's system some feedback statements are created in advance and stored in a data bank; 
they include fixed explanations to common errors. Other explanatory and remedial statements 
can be typed in by the tutor at the keyboard. 

Explanation lists are created by the program in advance according to whether things have been left 
out of students' answers, included in them correctly, or included but need improving. Finally, 
standard qualitative judgements and textual connectives are supplied by the program in the 
final feedback report. 

Marshall (1986/1987) 
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Strategy 6 

Avoid over-grading student 
work 



First consider, as a general rule, 

whether there might be occasions when rough grades, rather than precise, would be sufficient 
and acceptable, 

or whether rank order might on occasions do instead of individual grades, 

and whether analytic marking is always appropriate or necessary in the case of every 
assignment, 

and whether the marking criteria can be made more public, 

and by what means this can be done, in different parts of the course 

and on different occasions of assessment, 

and what might be gained or lost if more than one marker did the job. 
Then consider, in the particular assessment task being implemented, 

whether pass/fail, rank order, or grade clusters, might be possible, 
or whether a grade, rank or mark is really needed after all, 
and whether students know the marking criteria to be used, 
and, if not, whether they should, 

and whether anyone else is available for you to share the marking with. 

After these considerations, the items that follow may provide a source of ideas for you to use in 
starting to develop a more economical assessment system. 
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6.1 Making marking criteria public 

There is evidence that, in assessing written work, marker efficiency can be improved if both 
students and staff are aware of clear, explicit criteria, whatever the type of writing task. 

General criteria for what is acceptable and unacceptable in written work can be drawn up either on 
a School, Departmental, group or individual basis. 

Once drawn up, it must be contained in public documents distributed to students at the start of a 
course. 

6.2 Minimal marking 

Minimal marking is an economic approach for marking written work. It simply tells students 
there is an error in a certain passage or line and then requires them to identify and correct it. 
The teacher then comes in to assist only if students are unable to do this correcting. 

Minimal marking deals with "surface" writing faults only. It manages to refrain from marking 
these faults in the text itself, by using a simple, standardised code placed in the margin. The code 
draws students' attention to the number and type of faults in a particular line. 

When papers are later returned in class, students are given time to search for, circle and correct 
the errors before then handing the scripts back again. Only when this is done are grades finally 
recorded. 

Minimal marking is claimed to help "shorten, gladden and improve" the otherwise thankless task 
of marking written papers. 

Haswell (1983) 

6.3 Team marking 

Marking by team or panel can be more efficient than using a single marker. And provided 
evaluation criteria are carefully spelled out in advance, it can be very reliable. 

Efficiencies arise mainly through task-sharing and specialisation. To achieve this, a marking 
team divides examination questions among themselves, each person concentrating on only one or 
two across the whole class. They then develop greater speed, reliability, and a clearer sense of 
relative values of answers than if one person had to mark a number of different questions alone. 

It is both feasible and legitimate for marking panels to include people who have not been involved 
in teaching the course. However in such cases explicit criteria sheets (such as assignment 
attachments, Item 5.2) are obligatory. 
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Criteria sheets help ensure more accurate and efficient marking by the "outside" staff, who 
may often be graduate students. The critena statements also greatly facilitate the often time- 
consuming task of re-marking and going back to check on disputed marks. 

Gardner & Abraham (1978) 

6.4 Re-worked coursework 

Students can be given a chance to re- work coursework before it gets issued a final grade. That 
means giving students a second-chance and awarding grades on the final, rather than the first- 
submitted, version of a paper. This policy need not involve teachers themselves devoting more 
resources to marking. 

Economy is achieved when student peer-appraisal based on seminar groups is used at all stages 
prior to the last, when the final product is submitted (see Strategy 8 below for examples of 
peer-assessment). 

Cowan (1980) 

6.5 Machine-readable answer sheets 

Computer marking is the most efficient method for use with MCQs and other objective test 
formats where the answer can be chosen on a limited scale such as A-E. 

Computer marking requires that a machine-readable answer foim be used. 

Gibbs etal (1986) 67-69; Sommerfeld (1981) 88 

6.6 Computer-marking in laboratories 

Computer-marked laboratory practicals are possible in some instances. The goal of this strategy 
is to provide the most precise measures of accuracy in science and technology laboratory work. 
However their use has obvious benefits in economy of staff marking and grading time. 

Students enter their results at the keyboard and test their own answers against the programmed 
criteria, getting immediate feedback without staff time being used up (see also Items under 
Strategy 4 above). 

Gibbs et al (1986) 87-89; Grebenik (1985) 
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6.7 Pass/fail grading - for and against 

The most widespread, conventional grading system in universities is norm-referenced 
(sometimes called "grading on the curve"). In this approach grades refer to the performance of 
candidates relative to others in the group. 

Under such systems it has been suggested that economies in marking and scoring could be 
achieved if, instead of awarding percentage scores or a range of separate grades such as A, B, C, D, 
etc, papers might simply be marked as "pass" or "fail". 

How well is this proposal supported educationally? The educational arguments advanced in its 
favour include: 

(i) the irrelevance of precise grades outside the classroom context, 

(ii) the empirical fact that precise grades arc very poor predictors of future success, 

(iii) the meaningless of fine grade divisions owing to poor marking practices, 

(iv) the fact that pressure to get higher and higher grades often forces students into bad, 
superficial study habits and into examination cheating. 

Against pass- fail grading, however, it is argued that: 

(i) motivation to effort and achieving excellence can suffer, 

(ii) the student has an incomplete record of achievement at the end, 

(iii) and students themselves frequently resist implementation of pass-fail approaches. 

The issue is clearly unresolved. Markers seeking greater efficiency by this means must 
accordingly make their choice on the relevance of these arguments to their particular case. 

Ebel (1972)188-190 
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Strategy 7 

Refine current policies and 
find ways of using present 
methods better 



First consider, as a general rule, 

whether you (and your colleagues) are aware of the origins of and justification for the full 
range of assessment policies in current use in your Department or School, 

and whether these policies might, at least in part, benefit from a re -thinking, leading to revision 
or refinement, 



and whether there exists any policy regarding how staff time spent in implementing 
assessment can be credited appropriately in relation to time spent in class, 

and whether some time devoted to critically discussing the policies and considering 
alternatives might achieve economies in the long run, at either a School, Departmental 
or Course level. 



Then consider, in the particular assessment task being implemented, 

whether your specific assessment policy could be sharpened, refined or redesigned to make 
it more economical, 



whether there may be ways of arranging this task so that the time you spend on it can get more 
of the credit it deserves, 



and whether the Department or School is providing you with the opportunities and 
resources you need to critically examine all assessment policies that need to be implemented 
from time to time. 

After these considerations, the items that follow may provide a source of ideas for you to use 
in starting to develop a more economical assessment system. 




SCED Paper 78 43 a -f Strategies for assessing students 

1 X 



Refine current policies and find ways of using present methods better 



7.1 Improving traditional marking methods 

When essay papers are to be marked, the choice is normally between "sorting" and "point-score" 
methods. When being "sorted", essays are judged holistically rather than analytically, read as 
quickly as possible and sorted on first reading into broad categories. 

A second, more careful reading is then given to every paper, pile by pile, to confirm that each 
does belong in that category. This reading concentrates attention on papers noted as problematic. 
Letter grades are finally assigned on the basis of the sorting categories. 

When many papers are to be read in a short time, the "sorting" method is generally regarded 
as more efficient and less tedious than the alternative "point -score" method, though not 
necessarily inferior to it. 

In the familiar "point score" or analytic method, each answer is analytically checked against 
a list of points that represent the contents of an ideal answer, and a total score calculated by 
adding together these points or partial scores. 

There is also, however, a third method which is often found useful for dealing with very large essay 
numbers. This involves the use of model answers whose function is to illustrate grade point 
"thresholds". 

Markers read each essay quickly and gain an overall impression in relation to the several models 
(commonly three), then assign a grade solely on that impression. 

Green (1975); White (1985) 114-120; Cockburn & Ross (1978) 35; Heywood (1988) 39-40 

7.2 Improving content-marking schemes 

If it is decided to employ analytic marking of either essay or extended-answer questions, 
success will depend upon construction of an appropriate, detailed marking scheme. Such a scheme 
would normally specify: 

which particular course content is being looked for, 

and how many marks each item carries. 

These point-score marking schemes should, if possible, be made available to students. If this is not 
possible, some alternative means should be used for communicating explicit criteria concerning 
what is expected of students in the task (see Items in Strategy 1 above). 

UTRC (undated) 3 3-3. 4; Bamford (1978); Gibbs et al (1986) 143-146 




SCED Paper 78 44 Strategies for assessing students 

42 



Refine current policies and find ways of using present methods better 



7.3 Developing samples and model answers 

When essay questions are used for assessment, a range of sample or model essay/assignment 
answers can be supplied to students in advance. A varied stock of such models takes time but can 
be built up over a couple of years by keeping (with permission) photocopies of the best work each 
Session. 

The marks of authorship are removed and the models filed permanently in an accessible place such 
as a Departmental library. 

Those who use this approach claim that nothing is better able to illustrate "this is how you do it" 
than these examples of actual work that have gained good marks or are exemplary in some manner. 

Nimmo (1977) 186; Bamford (1978) 

7.4 Rules to deal with plagiarism 

The marker who encounters substantial plagiarism in students' wort: can find, quite apart from 
problems raised by ethical issues, that such encounters tend to be very time-wasting. 

Resolving a single suspicion of plagiarism can involve much reading to distinguish between 
copied and original sections of work, plus perhaps library searches to find evidence for the 
borrowing of sources. 

But since plagiarism is a predictable problem in many courses, it comprises one more of the 
things that, for both economy and consistency, we should be armed and ready for at the start 
of the course. 

The best approach seems to be to develop very explicit, departmentally-supported rules regarding 
how plagiarism and regurgitating will be dealt with. These are supplemented with clear guidelines 
for how to avoid it and how to move towards independent thought as a student. 

Examples of such rules in particular disciplines are available from the UNSW campus 
and elsewhere. 

Hill (1979); Andresen & Bond (1978) 98-101 

7.5 Better analytic grading 

It is generally believed that analytic grading tends to be more time-consuming than holistic grading, 
(see above, Item 7.1) 

It need not necessarily be so, however. Some economies are available when using the analytic 
grading approach, provided strategies are adopted which do not sacrifice validity and reliability. 
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One economic technique is for ideas expressed in the essay to be separately graded with 
positive or negative scores which represent either their sophistication or the magnitude of the error 
they contain. 

To do this, however, a suitable schema must be drawn up beforehand and followed scrupulously. 
Cockburn & Ross (1978) 36; Brindley (1978); White (1985); Ebel (1972) 179-185 

7.6 Handling scores with spreadsheets 

Spreadsheet software can be used for efficiently storing and manipulating scores and is believed to 
lead to good economies of time once a system is in operation. 

The simplest and least expensive public-domain or shareware software may be adequate 
for this, though more sophisticated programs are necessary if complex statistical analysis is to be 
undertaken. 

The spreadsheet is the electronic equivalent of a sheet of paper ruled into columns and rows. 
It can contain names, subjects and scores, and it is a simple matter to program it to automatically 
calculate and enter totals, aggregates, means, standard deviations, rank order and scaled scores. 
Graphs can often be plotted directly using an associated "graphics program". 

Camden (1987) 

7.7 Recognition for assessment time 

If assessment really matters towards encouraging student learning, there is one associated goal 
that staff should work towards. That goal is to have Schools and Departments recognise the time 
staff spend in certain aspects of assessment. 

The most reasonable way of affording this recognition is by classing and counting it as part of 
timetabled work- load. 

Would this be a step towards economy? It may, and the argument for economy draws upon student 
learning benefits that derive from these assessment activities (see Strategy 1 above). 

There are two conspicuous instances where the case for this recognition is particularly compelling: 

(i) discussion of individual essays with students, after they have been marked 
by staff; 

(ii) interview-marking, in which each script is marked in conference with the student 
who wrote it. 
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These each clearly deserve a modest re-allocation of teaching time. Each is a strategy by 
which assessment is able to be turned into a powerful teaching tool, with benefits in the quality of 
learning. 

Such approaches to assessment justify reallocating time resources which would otherwise be 
spent in kinds of teaching activities that may, on the evidence, be far less productive of high 
quality student learning - such as some types of lecturing. 

Nimmo (1977) 185; Good (1978) 

7.8 Assessment workshops for staff 

Since assessment policy generally involves all members of a Department or School, it follows that 
all staff should be involved in a coordinated plan of periodically scrutinizing and considering 
proposals for change. 

Efficiency can be often improved when staff as a whole better understand what should be done, 
why it should be done, what can be done and how it can be done best. 

It is generally necessary to update all these issues on an annual basis, as staff, subjects and 
conditions change. The basic structure of a sound assessment policy may, once settled, 
nevertheless be able to operate for the life of a course. 

Upgrading staff on assessment policy and practice is probably best achieved by spending a day or 
two in an intensive workshop. The basic agenda may include discussions at a general level 
exemplified by the suggestions above. 

The workshop agenda should also give staff themselves a chance to share their own useful "tricks 
of the trade" for saving time - tricks that have come from their own experience and which they know 
work. 

Among ideas already in use by staff and able to be shared by them if given the opportunity, 
may be found: 

assignment attachments, 
taped commentary, 
fast grading strategies, 
providing model answers. 

Many of these are covered later in this document. (See Strategy 8 below) 

It is especially important for tutors and other staff at all levels of appointment to be present 
at such meetings. The one qualification for participation should be that a person has a 
share in setting, marking, grading or discussing students' work. 
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Experience shows that meetings of this kind generally benefit from having a facilitator 
or chairperson from outside the specific subject discipline. 

Graff (1982); UTRC (undated) 3.1-3.4; Edwards (1979); Bligh etal (1975) 29-55; 
Cockburn & Ross (undated) 1-17; Gibbs et al (1986) 147-149; Kent (1989) 
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Strategy 8 

Consider alternative approaches 
to assessment policy and 
practice 



First consider, as a general rule, 

whether an adequately wide range of economic assessment alternatives is available or 
can be made publicly available for all staff in your Department or School to consider 
whenever they need to know about them, 

and whether staff are encouraged and given opportunity and time to consider, plus 
freedom to implement, economical assessment alternatives. 

Then consider, in the particular assessment task being implemented, 

whether it is one that could benefit from an alternative approach, 

and, if so, whether it might be worthwhile looking for an economical alternative in 
this Guide, 

and whether, if this Guide should contain nothing that appears to help, there might be 
anywhere else you could look to find the kind of alternatives that would suit you, your subject 
and your students. 

After these considerations, the items that follow may provide a source of ideas for you to use in 
starting to develop a more economical assessment system. 

8.1 Thesis and dissertation alternatives 

Increasing numbers of academics are considering and promoting alternatives to the extended 
Honours or Postgraduate Thesis/Dissertation. It is argued that the traditional forms may have 
outlived their usefulness in some subjects. 

Some alternatives have been proposed in recent literature. These should be examined to 
see whether, as they claim, they may be able to: 
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more fairly indicate the ability and potential of an Honours or graduate student, 
be more relevant to the sorts of tasks that will be done in subsequent employment, 
permit more efficient and economic supervision and marking practices. 
Stanton (1979); Reid (1978) 

8.2 Group-work alternatives 

Group Projects leading to a collaborative report can contribute to a particularly efficient use of 
limited marking time. 

There are, of course, problems associated with the awarding and distribution of group marks. 
Approaches have been developed for achieving fairness while rewarding individual excellence, 
particularly in subjects where practical tasks require team-work. 

One strategy used at the UNSW involves a collaborative group report being written by each 
laboratory syndicate. Each individual also submits a critical note indicating his or her degree of 
consent to (or dissent from) that report, with justifying reasons. 

Individual student grades comprise the group mark plus an appraisal of the quality of that 
extra note. Marking time is reduced substantially because only one report per group has to be 
assessed and the individual notes are relatively short. 

Ross (1978 ) 68; Gibbs et al (1986) 103-108; Andresen (1 979, 1982) Items on "Group Assessment" ; 
Magin (1982); Black et al (1977) 

8.3 Self-assessment alternatives 

There is good evidence that students gain from learning to evaluate their own progress. Self- 
assessment in one form or another has been shown capable of either supplementing or completely 
replacing teacher assessment. 

Students can in the first instance be involved in generating criteria upon which their work 
is to be assessed. Next they learn how to use these criteria to assign grades to both themselves 
and their peers. 

Like other skills self assessment needs to be practised, however, in order to be developed. It 
should never be used as a "once-off 9 thing, in isolation. It needs always to be embedded in a 
coherent training or induction program for students that forms an integral part of the 
academic course of study itself. 

Boud (1986); Stanton (1978); Bond & Lublin (1983) 93-122 
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8.4 Self-assessment in problem-based learning 

An example demonstrating how self-assessment may be carried out in respect of clinical 
reasoning or diagnostic skills in Medicine is given in detail in Barrows and Tamblyn. 

The system involves a simulated diagnostic setting. Skills being performed by trainees are 
video-taped, and students write up a report of the diagnostic procedure and their conclusions in a 
prepared format. They next sit for a bank of multiple-choice test items that are self-marked. 

Students then study a diagnostic assessment prepared by an expert physician for the same 
simulated patient, and compare it with their own account. Next they watch a video-tape of the 
experienced physician diagnosing the simulated patient. 

Finally students are shown a structured report on their own work filled in by the simulated patient, 
and view the video-tape of their own interaction with that patient. A standardised clinical skills 
evaluation form is then used to score all aspects of this complete test. 

This all takes about 1.5 hours. After it a student would review all the results and identify 
personal strengths and weaknesses. This then leads to the determination of what further 
information or skills need to be mastered. 

This fully-developed self-assessment model can serve as a very useful guide for staff unfamiliar 
with this approach. They could use it to develop their own self-assessment strategies in whatever 
problem-based disciplines they may work. 

Barrows & Tamblyn (undated) 41-44 

8.5 Self-assessment attachment sheets 

A simple way of introducing self-assessment at a modest level is to have a self-assessment sheet 
attached to work handed in for marking. 

The sheet will contain answers to a set of questions previously discussed, such as: 

"The good points about this piece of work are ..." f 

"The main weaknesses in this work are 

"What I would need to do to make it better is 

"This work deserves a mark of.., because ..." t 
... and so on. 



SCED Paper 78 



51 



Strategies for assessing students 



Consider alternative approaches to assessment policy and practice 



The technique contributes to student learning by encouraging students to be more self-critical 
of their own work. 

Other contributions to efficiency and economy can be got from these completed self-assessment 
attachments sheets, such as: 

the marker to more directly and immediately enter into the mind of the str Jent 
submitting the work, 

focus stronger attention on the work's strengths and weaknesses, 

address feedback comments that are meaningful and likely to be attended to. 

Stanton (1988); Oxford Polytechnic (1987) 

8.6 MCQ self-assessment with computer scoring 

Informal, optional, multiple-choice tests that are computer-marked are a means by which students 
can periodically assess their own progress through a course. Some departments offer facilities 
for this on, say, a fortnightly basis. 

Schemes of this kind can replace other periodic class-testing schemes, with substantial savings in 
staff time. Multiple-choice formats recommend themselves especially to mathematical and 
technical courses, but are often adaptable to other types of subject matter. 

For learning purposes it is not sufficient for students merely to discover whether or not they have 
selected the right option in an MCQ. 

A computer-based scheme can, however, supply them with all they need for a self-administered 
system: 

their answers) can be automatically checked, and 

a report sheet printed containing remedial advice on incorrectly answered questions. 
Jeffcoat & Yoong (1979) 

8.7 Peer-assessment alternatives 

Under appropriate circumstances, marks given by other students can be just as accurate and 
reliable as those given by lecturers. 

It is important, however, for the class group to generate marking criteria themselves and this can 
generally be done during tutorial time. 

In situations when lecturers are under pressure to evaluate student papers or assignments more 
quickly and superficially than they feel is desirable, a good case can often be made for "getting 
assistance" from students. 
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This step has the even more interesting implication of signalling a different relationship between 
lecturers and students in regard to assessment. It both presumes and contributes to a partnership or 
collaborative role rather than an adversary one. 

Peer assessment may also be used as a check of self-assessment. Staff mark only when needed to 
resolve discrepancies, or else make random checks to monitor the overall process. 

At the University of Newcastle peer-assessment takes place in a tutorial period after examinations. 
It is moderated by staff who are able to clarify the responses given in marking guides. Some limited 
amount of cross-marking is also possible here. 

Orpen (1982); Engel et al (1980); Feletti et al (1979) 

8.8 Peer-assessment of group work 

One model for alternative assessment in many disciplines is provided by a scheme for combining 
staff- and peer-assessment of students who have been undertaking team work. In this model, 
assessment is divided into two parts. 

First, a base mark is awarded by staff to all members of the team equally, based on the quality 
of the collaborative or joint product. 

To this is added an equally-weighted peer mark, acquired when each member of the team 
appraises each other's effort (but not his/her own). Specific guidelines and criteria are used for 
issuing this mark, which is then averaged for each team member. 

This assessment model has been used successfully both for oral presentations and for written 
reports. In addition to modelling the collaborative environment in which students will 
eventually have to work professionally, the approach carries evident marking economies for 
staff and has been recommended for other types of course (eg group design projects). 

Earl (1986); Fox (1985) 

8.9 Peer-assessment "pass the problem" strategy 

"Pass the problem" is a strategy especially applicable to a class comprising a small (tutorial size) 
group. 

A test paper is set containing the same number of problems or exercises as there are students in the 
group. Student 1 does problem 1 and passes it to student 2, who meanwhile has done problem 2 and 
passed it to student 3, and so on. Everyone has a chance to undertake one problem. 

They then have it marked by the student to whom it is passed and from whom they then get 
feedback on that problem. They also receive a problem worked by someone else, which they mark 
and to whom they give feedback. 

Stanton (1988); Oxford Polytechnic (1989) 
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8.10 Combined self- and peer-assessment 

If an assessment system enables students to assess both their own and each others' work this 
increases the amount of feedback they get on their own work while decreasing faculty work-loads 
at examination time. There are many possible strategies for arranging this. 

Under one scheme, examination papers have students' names removed immediately on collection, 
and are given a code number. Students mark one another's papers, anonymously, at home, using 
model solutions and marking guides, entering marks on a separate sheet (not the paper itself). 

Papers are redistributed to their authors (using the code) for self-maiking, and this marie is kept on 
a separate sheet. Papers and mark sheets are next sent back to the lecturer who inspects them and 
allocates a final mark. 

Students receive their own mark if the two marks differ by not more than 10% from one another or 
from the lecturer's mark. The student finally receives the work back, with marie sheets, and all 
identifications and codes removed. 

Such schemes require more preparation than usual before an examination and added clerical 
work (though a computer can help here). For small classes the net faculty load may be higher 
than otherwise. Trials show that the load becomes less, however, and the system more 
economical, as class size increases. 

Boud& Holmes (1981) 

8.11 Student-set questions and "learning teams" 

There is plenty of evidence that students are capable of writing usable exam questions themselves. 
This is a strategy offering obvious opportunities for economies of staff time. 

One model that gets them to do this is the "learning teams" approach. Learning teams are groups 
of students who meet - in or out of class time - to study together and test each other on course content 

Once such teams are operating they compose examination questions, pool them and sit for 
practice tests based on these questions. This leads to more lively discussion of subject matter 
outside class time, gives feedb: ,k on student progress and encourages more active learning. 

Questions generated in this way can later form the basis of an item pool for "real" examinations that 
are set and marked by staff and used for final grades and ':redentialing. 

Instructional Development at Waterloo (1987) 
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8.12 "Teach-testing" 

It is not always necessary for staff to spend time taking tests home or marking them outside class. 
A model that circumvents this is the 'Teach-Test" technique. 

Teach-testing involves a weekly class test being given routinely, during which the teacher tours 
the class, observing work and giving support and advice on request. Impromptu remedial 
explanations are given on the spot. 

The technique is most suitable for courses where clear objectives are specified and continuous 
assessment is not being used (Item 1 above). Its advocates claim, among its advantages, that it very 
effectively prepares students for final exams. 

Gibbs etal (1986) 165-166; Vickers (1984) 

8.13 Concept-mapping 

"Concept maps" have been reported to have been used successfully for a variety of purposes 
related to student learning, including assessment. 

A concept map is a device used to obtain a visualisation of the conceptual organisation the learner 
establishes for a given piece of knowledge. It is essentially qualitative, but could be adapted if 
desired to obtain a quantitative score relative to some standard. 

After a period of study, each student is asked to draw a concept map involving a set of ideas 
considered by the teacher to be key or constitutive of the section being studied. After drawing the 
map each student is briefly interviewed, during which he/she justifies the way the map has been 
organised and clarifies the meaning of the relationships established in it. 

The approach has been used for a variety of subjects, including "hard" sciences and technologies 
as well as English literature. Users of concept maps for evaluation of learning often recommend 
them as far superior to any other method of providing evidence of a student's understanding of 
subject matter. 

Once the basic techniques for drawing the maps have been taught, the economics of their use are 
very favourable. "Marking" time need involve only the short interview period during class time. 
Maps handed in can, however, provide a wealth of additional diagnostic information for the teacher. 

Moreira(1985) 

8.14 Grade-contracting by students 

The standards of the teacher as to what signifies "excellent" performance are among the most 
significant means by which teachers shape the academic values of the university. 
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•Contract Teaching' has become a generic name for arrangements whereby levels of perfoimance 
are set at the beginning of a course and work is completed when these standards have been met 
This "mastery leaming ,, approach is embodied in some well-known instructional systems 
eg Keller Plan, PSI, Audio-Tutorials. 

This means the teacher decides in advance what standards are appropriate for the grades of 
A, B, C etc, often in terms of certain options, eg test scores, written papers, projects, final 
exams. Within these limits students, at the start of term, propose their own patterns of work in the 
form of a written contract Tests and assignments are, within limits, able to be repeated until the 
agreed performance standards are met 

It is, however, generally admitted that this grading "by contract" can place a considerable extra 
load on the teacher, since clear course goals have to be stated, sequences and topics devised 
for attaining them, etc. Performance criteria consistent with course aims have also to be 
designated. Finally, at the assessment stage, considerable time has to be spent in discussion 
between lecturer/tutor and student. 

At face, none of this sounds like a recipe for economies. What is ignored by this criticism is, 
however, that mastery of learning systems such as this normally involve much independent 
student work with savings in the time staff would otherwise spend addressing classes. Lectures 
may be abandoned, for instance. 

Hence the additional time spent in having individual discussions with students needs to be seen as 
replacing, at least in part, time otherwise spent in preparing and delivering lectures, group tutorials, 
and other conventional teaching* events. 

It is possible that grade contracting, as part of a commitment to student independence, may, 
therefore, on balance represent a gain in efficiency for the teacher. Those interested should consult 
the companion volume to this Guide (#2 Strategies for Increasing Students' Independence), 
SCED Paper 77. 

Eriksen (1976); Todays Education (1977) 
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Strategy 9 

Consult original sources for 
how-to-do-it details 



First consider, as a general rule, 

whether copies of some of the papers referred to in this Guide are available in your School or 
Department as resources for staff to refer to, 

and whether there are particular assessment issues or concerns which the selected papers 
might need to address, taking into account the assessment problems peculiar to your School, 
Department or subject team. 

Then consider, in the particular assessment task being implemented, 

whether one or more of these papers might give you the detailed information you need to 
study an alternative assessment suggestion further, 

or tell you about ways to implement an alternative strategy or policy, 

and whether you ought, therefore, to call or phone your Professional Development Centre 
to get the information or advice you need. 
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AbOUt SEDA (formerly SCED) 



The Organisation: 

SEDA, the Staff and Educational Development Association, is the principal organisation in Britain for 
the encouragement of innovation and good practice in teaching and learning in Higher Education. SEDA 
is essentially a co-operative network of colleagues working with the following broad aims: 

• to lead and support effective improvements in the quality of students' educational experiences 

• to provide a forum for discussion and for collaboration on creative ideas for learning development 

• to assist the personal development of lecturers and educational developers 

• to encourage greater understanding of the nature of student learning 

• to offer support to new entrants to the profession 

• to enable the exchange of information and dissemination of good practice 

The organisation has grown from a loosely knit informal network of practitioners to its current status as 
a focus of activity in the field of educational development in Higher Education. This makes the 
administration more efficient and enables the exchange of information, dissemination of good practice and 
effective interaction of supportive colleagues, and recognises especially the need for universities and 
colleges to co-operate more closely. 

Conferences/Workshops : 

SEDA is highly regarded for its varied programme of events, designed to bring together practitioners of 
excellence in Higher Education for the interchange of ideas and ways of working. It holds at least two 
national conferences each year, together with regional conferences, workshops and special events. 

Consultancies: 

A major strength of the organisation is the number of highly experienced practitioners from universities 
and colleges who are able to initiate and run short courses, consultancies and workshops in a wide variety 
of fields such as initial training for new lecturers, gaming and simulations, group work, and all kinds of 
alternative approaches to teaching, learning and assessment. There is a register of SEDA members who 
can be contacted for these purposes. 

Local Networks: 

As the organisation has grown, local groups of SEDA practitioners have been meeting to arrange regional 
events, and in some parts of the country these have developed into organised local networks: there already 
exist effective local networks in the South-east, the North-east, the M1/M69 East Midlands group and in 
Scotland. Other groups are currently being set up in the West Midlands and Central England, and in the 
North-west and North Wales. 

Membership: 

There are three categories of membership: Associate, Individual and Corporate. 

Information: 

For further information about membership, or any other aspect of SEDA, contact SEDA's Administrator: 
Jill Brookes, Gala House, 3 Raglan Road, Edgbaston, Birmingham B 5 7RA Tel: 02 1 446 61 66 Fax: 02 1 446 599 1 
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